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(54) A dejittering and clock recovery technique for real-time audio/visual network applications 



(57) In a real-time audio/visual system in which A/V 
data is conveyed over a jitter-introducing network, dejit- 
tering and clock recovery processes can be achieved 
without requiring a Phase Locked Loop (PLL). At the 
server, audio/video streams are encoded into transport 
packets before being sent out. At the client, the dejitter- 
ing process is achieved by a dejittering buffer using the 
embedded timestamps in the transport packets and a 
client decoding clock. The delay variations of data arriv- 
ing are removed after the client buffering process. At the 



scheduled time, each data packet is shifted to a syn- 
chronizing buffer and then fed to the A/V decoder ac- 
cording to the speed of A/V stream. The clock synchro- 
nization between client and server is achieved by a syn- 
chronizing buffer whose half-size position is taken as the 
reference. By monitoring the movement of the buffer fill 
position over a given period, the drift rate of clock un- 
synchronization between client and server can be de- 
rived and, therefore, the client's clock can be adjusted 
to synchronize with the server's clock based on the de- 
rived drift. 
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[0006] It is an object of the present invention, to pro- 
vide a technique for dejittering and clock recovery for 
use in the client application of a networked real-time au- 
dio/visual service system. It is desirable to be able to 
implement embodiments of the invention to synchronize 
the clients to a server in a jitter introducing network en- 
vironment without employing additional devices or a 
special decoder. 

Summary of the Invention 

[0007] In accordance with the present invention, there 
is provided a method for clock variation compensation 
in a real-time audio/visual system in which encoded A/ 
V data in a plurality of data packets are delivered to at 
least one client over a network from a server at a sub- 
stantially constant bit rate, said plurality of data packets 
including data packets containing time stamp data the 
method comprising the steps of: 

receiving and buffering the data packets in a first 
buffer at a client; 

passing selected data packets from the first buffer 
to a second buffer at scheduled times based on a 
comparison between a local clock of the client and 
tim ©stamp data corresponding to the selected data 
packets; and 

passing the data in the second buffer to a data de- 
coder of the client. 

[0008] Preferably the method includes monitoring the 
fullness of the second buffer to derive a drift rate, and 
adjusting the client local clock based on the derived drift 
rate. 

[0009] The present invention also provides a method 
for clock variation compensation in a real-time audio/ 
visual system in which encoded A/V data in a plurality 
of data packets are delivered to at least one client over 
a network from a server at a substantially constant bit 
rate, said plurality of data packets including data pack- 
ets containing timestamp data, the method comprising 
the steps of: 

receiving and buffering the data packets in a dejit- 
tering buffer at a client; 

passing selected data packets from the dejittering 
buffer to a data decoder of the client at scheduled 
times based on a comparison between a decoding 
system clock of the client and timestamp data cor- 
responding to the selected data packets. 

[001 0] Preferably the method includes monitoring the 
fullness of the dejittering buffer to derive a clock drift 
rate, and adjusting the decoding system clock based on 
the derived drift rate and a network jitter component. 
[0011] The present invention further provides a real- 
time audio/visual system coupled to receive data pack- 
ets over a network for A/V decoding by an A/V decoder, 



the system including a clock variation compensation 
system comprising: 

a dejitter buffer for receiving and storing packets of 
s data from the network; 

a synchronization buffer for feeding data for decod- 
ing to the AW decoder; 
a decoder system clock; and 
a buffer data flow controller for controlling the pass- 
ing of selected data packets from the dejitter buffer 
to the synchronization buffer in accordance with a 
comparison of a first signal derived from the decod- 
er system clock and a second signal derived from 
a timestamp from the selected data packets. 

[0012] The technique summarized below is a 'soft- 
ware PLL-like" method to address the jitter and time syn- 
chronization for a client decoding system. It employs the 
RTP (Real-Time Transport Protocol) as the transport 
service and receiver buffering to achieve real-time A/V 
playback. 

[0013] The dejittering process can be achieved by a 
dejittering buffer using the embedded timestamp values 
in the transport packets and client RTP clock (which 
runs at the same frequency as the A/V decoder's clock), 
The delay variations of data arriving are removed after 
the client buffering process, the data packet is shifted to 
a synchronizing buffer at the scheduled time of server 
encoder, then feed to the A/V decoder according to the 
time reference of A/V encoder. The clock synchroniza- 
tion (recovery) can be achieved by a synchronizing buff- 
er based on a reference fill position and the movement 
of the fill position in the buffer (packet index oriented). 
By monitoring the fullness of the buffer over a given pe- 
riod, the drift rate of clock unsynchronization between 
client and server can be derived. 

Brief Description of the Drawings 

[0014] The invention is described in greater detail 
hereinafter, by way of example only, with reference to 
embodiments thereof which are described with the aid 
of the accompanying drawings, wherein: 

Figure 1 schematically illustrates a networked sys- 
tem for real-time audio/visual services comprising 
a plurality of client host and a server: 
Figure 2 schematically illustrates a client host com- 
prising a dejittering buffer and a software PLL-like 
architecture, wherein dejittering and clock recover- 
ing is achieved in accordance with an embodiment 
of the present invention; and 
Figure 3 diagrammatically illustrates the mecha- 
nism of timing synchronization via monitoring the 
position movement of the dejittering buffer 
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with the reference position (defined as the buffer half- 
size position) for a given period 7" (lor example, every 
minute). If the buffer position is moving in the direction 
of emptiness, the counter TC should be upwardly ad- 
justed by adding an offset. If the buffer position is moving 
in the other direction (fullness), the TC value should be 
downwardly adjusted. The drift rate r of clock unsyn- 
chronization between server and client can be deter* 
mined by: 

r = (P 2 -p 1 VT 

[0023] It should be noted that the buffer position men- 
tioned above is in terms of packet index offset rather 
than the byte number offset. 

[0024] It is also possible to implement the present 
technique with the use of one buffer only (no additional 
synchronizing buffer available). In such case, data pack- 
ets are fed into the A/V decoder directly from the de jit- 
tering buffer at the scheduled time. The above clock drift 
rate /will include the component of network jitter J and 
should be eliminated before the TC is adjusted. The in- 
terarrival jitter J can be derived from the two sequenced 
RTP packets. The jitter J is defined to be the mean de- 
viation of the difference (D) in packet spacing at the re- 
ceiver compared to the sender for a pair of packets. For 
example, if 7" OT is the RTP timestamp from packet a, and 
Tfr is the time of arrival in RTP timestamp units for pack- 
et b, then for these two packets, we have, 

j=D(a,b) 

[0025] The j is calculated continuously as each data 
packet is received from server, then according to the for- 
mula, we have, 

J=j+(j- j;/i6 

[0026] This algorithm provides an optimal first -order 
estimator and ihe gain parameter 1/16 gives a good 
noise reduction ratio while maintaining a reasonable 
rate of convergence (see RTP specification and related 
references). 

[0027] Unlike the traditional method (using a PLL cir- 
cuitry) and other available technique (such as that dis- 
closed in European patent No. EP779725, entitled 
"Method and Apparatus for Delivering Simultaneous 
Constant Bit Rate Compressed Video Streams At Arbi- 
trary Bit Rates with Constrained Drift and Jitter", which 
uses two levels of synchronization named coarse -grain 
and fine-grain for video streams to control drift and jit- 



ter), the present method is a more independent and less 
constrained client-based approach. It provides the net- 
work adaptability via a simple software solution (can be 
implemented in hardware as well) for an end system and 
£ can be applied in a wide range of network applications. 
The effect of the present technique includes: 

increasing the adaptability of decoding systems (in 
terms of dejittering and clock synchronization); 
io simplifying the decoder implementation (no PLL cir- 
cuitry required); 

handling more types of audio/visual streams (not 

only MP EG 2 system streams); 

application to different types of environments (uni- 

cast as well as multicast); 

beneficial to the effort of expanding real-time A/V 

services over IP-based network such as Internet. 

[0028] The foregoing detailed description ol embodi- 
ments and implementations of the invention have been 
presented by way of example only, and is not intended 
to be considered limiting to the present invention as de- 
fined in the claims appended hereto. Numerous alterna- 
tive embodiments may be devised by those skilled in the 
art without departing from the spirit and scope of the in- 
vention. 

[0029] Throughout this specification and the claims 
which follow, unless the context requires otherwise, the 
word "comprise", and variations such as "comprises" 
and 'comprising", will be understood to imply the inclu- 
sion of a stated integer or step or group of integers or 
steps but not the exclusion of any other integer or step 
or group of integers or steps. 



Claims 

1. A method for clock variation compensation in a real- 
time audio/visual system in which encoded A/V data 

40 in a plurality of data packets are delivered to at least 
one client over a network from a server at a sub- 
stantially constant bit rate, said plurality of data 
packets including data packets containing times- 
tamp data, the method comprising the steps of: 

45 

receiving and buffering the data packets in a 
first buffer at a client; 

passing selected data packets from the first 
buffer to a second buffer at scheduled times 
£0 based on a comparison between a local clock 

of the client and timestamp data corresponding 
to the selected data packets; and 
passing the data in the second buffer to a data 
decoder of the client. 

55 

2. A method as claimed in claim 1 , including monitor- 
ing the fullness of the second buffer to derive a drift 
rate, and adjusting the client local clock based on 
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(54) A dejittering and clock recovery technique for real-time audio/visual network applications 



(57) In a real-time audio/visual system in which A/V 
data is conveyed over a jitter-introducing network, dejit- 
tering and clock recovery processes can be achieved 
without requiring a Phase Locked Loop (PLL). At the 
server, audio/video streams are encoded into transport 
packets before being sent out. At the client, the dejitter- 
ing process is achieved by a dejittering buffer using the 
embedded timestamps in the transport packets and a 
client decoding clock. The delay variations of data arriv- 
ing are removed after the client buffering process. At the 



scheduled time, each data packet is shifted to a syn- 
chronizing buffer and then fed to the A/V decoder ac- 
cording to the speed of A/V stream. The clock synchro- 
nization between client and server is achieved by a syn- 
chronizing buffer whose half-size position is taken as the 
reference. By monitoring the movement of the buffer fill 
position over a given period, the drift rate of clock un» 
synchronization between client and server can be de- 
rived and, therefore, the client's clock can be adjusted 
to synchronize with the server's clock based on the de- 
rived drift. 
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